Design of a Questionnaire Instrument

Questionnaires have obtained a rather ambivalent reputation as a research instrument. Although they can be very useful, designing a good questionnaire takes considerable effort and thinking. Two things that are not always around, it seems. Not surprisingly, we all have come across some, to put it mildly, terrible questionnaires. Sitting down for five minutes to write down a number of questions is, of course, unlikely to result in a brilliant questionnaire. Still for the untrained eye, it looks like a questionnaire and again another proof that they should not be trusted. Among Human-Computer Interaction (HCI) professionals, this aversion is also noticeable. For example, when asked to compare usability methods, professionals rated questionnaires and surveys among the bottom of methods they use or have used (Gulliken et al., 2004; Bark et al., 2005; Mao et al. 2005). On the other hand, a large group is using them. For example, in a survey among 197 HCI practitioners (Bark et al. 2005) 53% of the responders had used a questionnaire to evaluate a product, whereas 97% had used a user test for this. This means that although HCI professionals might sometimes be sceptical towards questionnaires, they do use them and therefore there is a market for good questionnaires. In this chapter I will try to give you an insight in how to design, evaluate, and administer such a questionnaire. I will do this by providing examples from research I have done with my colleagues over the years in the area of mobile HCI. From reading this chapter you should be able to setup a questionnaire, conduct various analyses to check its reliability, its validity, and have an idea how to distribute and analyse the questionnaire.